A continuation method for solving separable nonlinear least squares problems

Randomized Algorithms for Solving Large Scale Nonlinear Least Squares Problems

This thesis presents key contributions towards devising highly efficient stochastic reconstruction algorithms for solving large scale inverse problems, where a large data set is available and the underlying physical systems is complex, e.g., modeled by partial differential equations (PDEs). We begin by developing stochastic and deterministic dimensionality reduction methods to transform the ori...

Newton-Krylov Type Algorithm for Solving Nonlinear Least Squares Problems

The minimization of a quadratic function within an ellipsoidal trust region is an important subproblem for many nonlinear programming algorithms. When the number of variables is large, one of the most widely used strategies is to project the original problem into a small dimensional subspace. In this paper, we introduce an algorithm for solving nonlinear least squares problems. This algorithm i...

Solving Derivative-Free Nonlinear Least Squares Problems with POUNDERS

Supervised Descent Method for Solving Nonlinear Least Squares Problems in Computer Vision

Many computer vision problems (e.g., camera calibration, image alignment, structure from motion) are solved with nonlinear optimization methods. It is generally accepted that second order descent methods are the most robust, fast, and reliable approaches for nonlinear optimization of a general smooth function. However, in the context of computer vision, second order descent methods have two mai...

An Efficient Algorithm for the Separable Nonlinear Least Squares Problem

The nonlinear least squares problem miny,z‖A(y)z + b(y)‖, where A(y) is a full-rank (N + `)× N matrix, y ∈ Rn, z ∈ RN and b(y) ∈ RN+` with ` ≥ n, can be solved by first solving a reduced problem miny‖ f (y)‖ to find the optimal value y∗ of y, and then solving the resulting linear least squares problem minz‖A(y∗)z + b(y∗)‖ to find the optimal value z∗ of z. We have previously justified the use o...